Assessing and Improving the Reliability of Volunteered Land Cover Reference Data

Volunteered geographic data are being used increasingly to support land cover mapping and validation, yet the reliability of the volunteered data still requires further research. This study proposes data-based guidelines to help design the data collection by assessing the reliability of volunteered data collected using the Geo-Wiki tool. We summarized the interpretation difficulties of the volunteers at a global scale, including those areas and land cover types that generate the most confusion. We also examined the factors affecting the reliability of majority opinion and individual classification. The results showed that the highest interpretation inconsistency of the volunteers occurred in the ecoregions of tropical and boreal forests (areas with relatively poor coverage of very high resolution images), the tundra (a unique region that the volunteers are unacquainted with), and savannas (transitional zones). The volunteers are good at identifying forests, snow/ice and croplands, but not grasslands and wetlands. The most confusing pairs of land cover types are also captured in this study and they vary greatly with different biomes. The reliability can be improved by providing more high resolution ancillary data, more interpretation keys in tutorials, and tools that assist in coverage estimation for those areas and land cover types that are most prone to confusion. We found that the reliability of the majority opinion was positively correlated with the percentage of volunteers selecting this choice and negatively related to their self-evaluated uncertainty when very high resolution images were available. Factors influencing the reliability of individual classifications were also compared and the results indicated that the interpretation difficulty of the target sample played a more important role than the knowledge base of the volunteers. The professional background and local knowledge had an influence on the interpretation performance, especially in identifying vegetation land cover types other than croplands. These findings can help in building a better filtering system to improve the reliability of volunteered data used in land cover validation and other applications.